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ABSTRACT 

The school self-Jtudy process for participants who 
had been through it before was compared with the process in schools 
new to self-study using the "K-12 School Evaluative Criteria" (1983). 
Self-studies had been completed by pilot schools in the 1981-82 
school year. Personnel from 11 schools with a previous self -study and 
123 participants with no previous experience answered questionnaires 
about the instrument anc were compared using t-tests for their 
opinions of: (1) adaptability; (2) clarity; (3) effectiveness in 
determining the degree of articulation; (4) grammatical structure; 
and (5) terminology. No significant differences existed between 
experienced and inexperienced participants' ratings. Both novice and 
experienced participants also considered that a kindergarten through 
grade 12 stelf-study was more beneficial than a specific level study. 
Previous experience did not apparently improve the school's ability 
to cope with the process, although those with previous experience did 
begin making curricular changes more rapidly. Statistical data are 
suimmarized in tables. (SLD) 
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The Second Time Around: Are Self-Studies Easier and 
Do They Have More Impact if School's Personnel 
Have Been Through the Process Before? 

Evaluation has become a word of major importance on today's 
educational scene. With accountability no longer considered educa- 
tional jargon but rather an essential word in one's vocabulary, the 
public is demanding to see evidence of well planned, implemented and 
evaluated curriculum designs. Likewise, as the public tends to be 
skeptical of the ability of some teachers, school systems are beginning 
to systematically review their strengths and weaknesses with the 
public's help in order to improve the perceptions of education that the 
general populace now holds. The Evaluation of Secondary School ' s 
(General Report) (Cooperative Study of Secondary School Standards, 
1939, p. 366) said, "By what means and processes does a good school 
develop into a better one?" The answer given is "through diagnosis of 
its weaker points, and stimulation to improvement, followed by 
appropriate action in line with such diagnosis." In ot er words, by 
evaluation! The traditional view of evaluation has been a process 
which enables data to be gathered, analyzed, and synthesized in order 
to enable decision making ^Borich & Jamelka, 1981). Cronbach, Ambron, 
Dornbusch, Hess, Hornik, Phillips, Walker, and Weiner (1980) defined 
evaluation as a "systematic examination of events occurring in and 
consequent on a contemporary program — an examination conducted to 
assist in improving this program and other programs having the same 
general purpose" (p. 14). Cronbach et al. claimed that the field of 
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evaluation has "become the liveliest frontier of American soc-5i=il 
science" (p. 13 ) . 

When speaking of evaluation, Eisner (1979) has found that two 
central functions exist: 1) to make revisions in the cvrriculum, and 
2) to identify and anticipate educational needs. He has condoned the 
notion that unless the quality of the components of the curriculum are 
appraised, therf* is no conceivable means to determine its worth. He, 
too, has promoted the appropriateness of the curriculum for the 
intended population . 

Saylor, Alexander, and Lewis (1981) avowed that the basic goal of 
evaluation is the ability to determine the worth or value of something. 
More specifically, 1) to determine if the curriculum meets the intended 
purposes, 2) if those purposes are valid, 3) if the curriculum is 
appropriate for the clientele it serves, and 4) if so, are the 
instructional models, the content, and the instructional materials that 
are selected pare llel with the purposes? 

Trump and Georgiades (1975) have specified the major aim of 
evaluation is the ability to determine how a given educational program 
is better than it was previously and how more improvements may take 
place in the future. For this reason any evaluation must strive to 
collect comprehensive data regarding all aspects of the school. 

But what data are needed and how are rhe data collected at school? 
For the most part, schools* personnel complete self study materials. 
The Cooperative Study of Secondary School Standards, now called the 
National Study of School Evaluation, began developing such self study 
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materials for secondary school evaluations back in 1933. Over the 
years, middle/ junior high schools and ele^^entary schools have been 
provided with evaluation instruments, as well. But, until recently, 
the unit schools, which house grades K-12, had no such mechanism to 
systematically view the total school program. So in 1980, the 
Executive Director of the National Study of School Evaluation, Dr. 
Donald C. Manlove, made a proposal regarding the feasibility of an 
evalurition instrument that would answer requests made for a K-12 
evaluation tool. The Board of Directors, deciding there was a void 
where this specific type of evaluation was concerned, directed Dr. 
Manlove to start the developmental process for an instrument to meet 
the needs of K-12 unit schools. Having had much experience in other 
developmental projects, the Executive Director was able to begin the 
effort expedi tiously . 

A National Advisory Committee was selected. It was composed of 
members from all regional accrediting associations; these educators 
had expertise in the areas that would be covered in the K~12 
instrument. Several people had in-depth curriculum backgrounds and all 
members had classroom and/or administrative experience in elementary, 
middle and/or high schools. Committee members represented public, 
private and independent schools. The diversity of the group enabled 
the team to serve the K-12 schools in a professional and accurate 
manner. 

The National Advisory Committee first met in St. Louis, Missouri, 
in January of 1981, At this time the materials for the new instrument 
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were assembled* While this committee was requested to draw hea'*ily up- 
on other NSSE materials, the team was also asked to include only items 
specifically apropos to a K-12 situation. Naturally, any new concepts 
had to be addressed and therefore, many new items had to be created for 
such an instrument. 

The regional accrediting associations were notified of the NSSE's 
developmental effort and asked to submit names of K-12 schools that 
would be Involved in self studies for the 1981-82 school year. Prom 
this compilation, twelve schools were selected and Invited to partici- 
pate; thjis enabled these schools to undergo self studies on a complete 
K-12 scale, using only one instrument. In the summer of 1981 this 
writer became involved with the project in the capacity of the Research 
Assistant . 

During the 1981-1982 school year, the pilot schools were involved 
in their self studies. As they were engaged in their subcommittee 
meetings, they critiqued the instrument by writing their comments on 
one copy of the proposed K-12 School J Svaluat ive Criteria, Upon (or 
near) completion of the self study, the participants were asked to 
complete a questionnaire. 

But as the self studies were taking place numerous questions were 
arrlsing from the schools' personnel. It became of interest to see how 
the perceptions of the self study participants who had been through the 
process before compared with the participants who were new to the 
process. Since, at the time of this study, an instrument had not yet 
been available which examined the total school program, many K-12 
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schools that had desired to undergo a self study had not gotten 
involved since technically they would have had to use three separate 
instruments for the various organizational divisions (i,e, elementary, 
middle, high school). This would have broken the continuity desired 
in K-12 schools. 

Now, schools never before involved in self studies are using the 
new K-1 2 School Evaluative Criteria (1983) , Of course, schools that 
had taken part in previous self studies are also using the new 
document, A difference existed, however, between previous self studies 
in which the schools had participated and the pilot test of this 
instrument: the teachers had only participated in self studies with 
members from similar backgrounds (elementary, middle, or high school). 
For this reason, it became a concern of the project director and this 
researcher if using this type of self study instrument would create 
more problems for the faculties with previous self study experienca or 
those without such experience. 

One of the twelve pilot schools was unable to complete its self 
study so this study represents a 264 (70^) return of the annonymous 
questionnaires from personnel from eleven schools: Of these, 141 
participants had previous self study experience and 123 had no previous 
experience. The statistics for four questionnaire items were placed 
into their appropriate categories determined by whether or not the 
faculties at the schools had participated in self studies before. 
These two groups were compared using frequencies and percentages on 
three items and using t-tests on their ratings of five characteristics 
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of the instrument . 

The first question examined was an item dealing with the partici- 
pants* rating of the instrument. The areas under examination were: 
1) adaptability; 2) clarity, 3) effectiveness in determining the degi ^e 
of articulation, 4) grammatical structure, and 5) terminology. For the 
purposes of calculating, the ratings were assigned numerical values. 
In thif5 case. Excel lents54 , Good=:3, Fair=2, and Poor=l. Those choosing 
to make no comment were not calculated in this 5~part item. The means 
and variances were found for the experienced and Inexperienced 
participants in all five areas. They were then compared using t-tests. 
The formula used to calculate the t-tests was 

Xold - Xnew 

/ (Nold-1) » o^ld + (Nnew-1) ♦ cThew Nold -h Nnew 
No Id + Nnew -2 Nold • Nnew 

Tho results may be seen in Table 1. 

Insert Table 1 here 

In all five areas t=<.05. Therefore, the results indicate that no 
significant difference existed between the experienced and the in- 
experienced participants* ratings of these five areas. 

These data indicate that the participants shared similar senti- 
ments regarding the instrument. Therefore, having experience with 
other self studies, normally using other NSSE instruments, did not 
cause the participants to view the process in a different manner than 
neophyte participants. While one might wonder why experienced 
participants would not have given the instrument higher ratings, 
especially in the areas of adaptability and claiity, the additional 
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difficulty of coping with many other grade levels in this self study 
may have created the lack of discrepancy. 

It would appear from these data that K-12 schools that never 
before have used self study materials need not fear plunging into this 
new instrument. These data statistically show that faculties with self 
study experience and faculties without such experience perceived the 
instrument in much the same manner. 

Next, a questionnaire item regarding the benefit of a K-12 self 
study over a specific level self study was compared. The cat*?gorized 
results appear in Table 2. 

Insert Table 2 here 

The results indicated that both the novice participants and the 
experienced participants found a K-12 self study was of greater benefit 
than a specific level self study would have been. As the discrepancy 
between the two groups was examined, it became apparent that both Yes 
and No responses received almost identical percentages, in fact, the 
results indicated a mere discrepancy of one percentage point. 

To delve furtb'^r to see if a discrepancy existed between faculties 
with self study experience and faculties without previous self study 
experience, the participants' responses to a question regarding I'he 
resultant continuity in the curriculum were compared. The results are 
tabulated in Table 3. 

Insert Table 3 here 
Eighty-six (61*) of those responding from the schools with 
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experienced participants said that curricular changes had taken place 
at or near the completion of the self study with 60 (495K) saying this 
from neophyte schools. It appears that the schools having had previous 
experience with self studies tended to begin making curricular changes 
more rapidly. It is interesting to note that a total of 19 {13%) of 
the respondents from the experienced schools withheld judgment while 28 
(22%) withheld judgment from the newly participating schools. A bit of 
skepticism may be noted here on the part of the neophytes but no 
significant problem is apparent. 

A fourth item on which the groups were compared was the adequacy 
of the proposed instrument. Table 4 represents the tabulated results. 

Insert Table 4 here 

Both groups responded to this item in similar fashions. While t 
experienced group seems slightly more positive, all results had a 
discrepancy of five percentage points or less. Again, the neophyte 
faculties haa a greater percent of people choosing not to comment. 

Despite the fact that certain facets of the instrument needed 
improvement, overall, the instrument, and the K-12 philosophy behind 
the instrument, appeared to be sound since both groups were in close 
agreement on all items; all of these compared questions indicated there 
was no significant difference in the responses o: the participants from 
these two groups. Again, this lack of a significant discrepancy 
indicated that the teachers from schools never before undertaking a 
self study were able to cope with a self study with no more difficulty 
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than teachers from any other schools. These data support the notion 
that each self study was a totally new experience. Again, it should be 
noted that those faculties which had previous experience had never 
before used the exact instrument. In addition, numerous new people sat 
on the subcommittees since each subcommittee had to represent the 
entire K-12 spejtrum. Such changes may have had an effect on the lack 
of significant discrepancies since even the experienced participants 
were fording into some previously unexplored areas. 

The data were analyzed and presented to the National Advisory 
Committee; the internment w-vS then refined and a glossary of terms was 
added to clarify terminology. The instrument was published in 1983 and 
has been used by numerous K-12 schools. 

In a few years, after several schools have used the K-12 School 
Evaluative Criteria and those schools are ready to undergo re- 
evaluation, it would be interesting to duplicate this study. In the 
duplication study, however, the experienced participants should be 
those who have used the exact instrument on a K-12 bases. 
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Table 1 
Ratings of tne Instrument 





Experienced j 


Unexperienced j 








Mean 


Variance j 


Mean 


Variance j 


T 


-test 


Adaptability to | 
Your Particular j 
School 1 


2.784 


.419 1 


2 . 692 


.275 1 


1 


.216 


Clarity | 


2 .330 


.566 1 


2 . 172 


.415 1 


1 


.769 


Effectiveness in j 
Determining Degree j 
of Articulation j 


2 .527 


.792 1 


2 .533 


. 572 1 




.0529692 


Grammatical j 
Structure j 


2.837 


.779 1 


2 .774 


.334 1 




.655 


Terminology j 


2 .451 


.471 1 


2 .355 


.489 1 




.092 
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Table 2 

Is Having a K-12 Self Study Rather Than a Specific 
Level Self Study More Beneficial to 
the School District? 



Response j 


Schools with Previous 
Self Study Experience 


j Schools 
1 Study 


with No Self 
Experience 




Number j Percent 


1 Number 


1 Percent 


Yes 1 


103 1 73 


1 89 


1 72 


No 1 


30 1 21 


1 ''5 


1 20 


Undecided | 


5 1 4 


1 8 


1 7 


No Comment j 


3 1 2 


1 1 


1 1 


Total 1 


141 1 100 


1 123 


1 100 
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Table 3 

Does the School Curriculum Have More Continuity 
as a Result of the K-12 Self Study? 



Response j 


Schools with Previous 
Self Study Experience 


j Schools 
1 Study 


with No Self 
Experience 




Number 


Percent 


j Number 


I Percent 


Yes 1 


86 


61 


1 60 


1 49 


No 1 


36 


26 


1 35 


1 29 


Undecided j 


10 


7 


1 


1 1^ 


No Comment | 




6 


1 


1 


Total 1 


141 


100 


1 123 


1 100 
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Table 4 

Is the Proposed Instrument K-12 Evaluative Criteria Adequate for 
Evaluating Quality Education in K-12 Schools? 



Response 


Schools with Previous 
1 Self Study Experience 


j Schools 
1 Study 


with No Self 
Experience 


1 Number 


■ 
I 

1 Percent 


1 Number 


1 Percent 


Yes 


49 


1 35 


1 38 


1 31 


No 




1 ^ 


1 d 


1 


Yes, with 
Revision 


74 


1 52 


1 58 


1 


Undecided 


0 


1 0 


1 1 


1 1 


No Comment 


12 


1 d 


1 17 


1 


Total 


141 


1 100 


1 123 


1 100 



16 



